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Abstract. We address the problem of chaotic temperature dependence in disordered glassy systems at 
equilibrium by following states of a random-energy random-entropy model in temperature; of particular 
interest are the crossings of the free-energies of these states. We find that this model exhibits strong, 
weak or no temperature chaos depending on the value of an exponent. This allows us to write a general 
criterion for temperature chaos in disordered systems, predicting the presence of temperature chaos in 
the Sherrington-Kirkpatrick and Edwards- Anderson spin glass models, albeit when the number of spins is 
large enough. The absence of chaos for smaller systems may justify why it is difficult to observe chaos with 
current simulations. We also illustrate our findings by studying temperature chaos in the naive mean field 
equations for the Edwards- Anderson spin glass. 



PACS. 75.50.Lk Spin glasses and other random magnets 
64.70.Pf Glass transitions 



05.70.Fh Phase transitions: general studies 



1 Introduction 

It is generally agreed that the equilibrium states of spin 
glasses [QU are "fragile", i.e., they are sensitive even to 
very small perturbations. Roughly, this can be argued 
from the fact that there exist many (meta-stable) states 
whose excess free-energies grow more slowly than the sys- 
tem's volume; then arbitrarily small extensive perturba- 
tions will re-shuffle the different (meta-stable) states and 
the lowest free-energy state will be completely different 
from the one without the perturbation. This phenomenon 
is referred to as "chaos" || as the equilibrium state de- 
pends chaotically on the perturbation. In particular, there 
has long been a consensus that chaos arises when changing 
the couplings Jy between the spins. However, the situa- 
tion is quite different when the parameter being changed 
is the temperature; though earlier work [^,|^] claimed that 
there is chaos under even infinitesimal temperature changes, 
some recent work goes against this; indeed, analytic ex- 
pansions (^] around the critical temperature give no tem- 
perature chaos in the Sherrington-Kirkpatrick (SK) model, 
and extensive Monte Carlo simulations in both the three- 
dimensional Edwards- Anderson (EA) model and the SK 
model find little or no evidence of temperature chaos |7|, 

This controversy is interesting in itself as it shows just 
how poorly understood spin glasses remain after years of 
work. More importantly perhaps, temperature chaos plays 
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a central role in many theoretical descriptions of dynam- 
ics. Indeed, phenomena such as rejuvenation || that follow 
temperature changes in glassy systems must arise if there 
is temperature chaos (see for instance [H and jll| ) . Thus 
both for equilibrium and for non-equilibrium properties, it 
is appropriate to understand whether or not there is tem- 
perature chaos in disordered systems, and in spin glasses 
in particular. 

In this work, we present a solvable glassy model where 
the temperature dependency is easily analyzed. Our main 
result is that the presence of temperature chaos depends 
on an exponent associated with state-to-state entropy fluc- 
tuations. Depending on the value of this exponent, the 
model exhibits strong, weak, or no chaos at all. When ex- 
trapolating these results to realistic models, we find chaos 
both for the SK and EA models but only when the number 
of spins N is quite large, 0(1000). This may justify why 
chaos is not seen in Monte-Carlo simulations. Finally, we 
attempt to confirm numerically these predictions by using 
a naive mean field approximation; there we also find the 
expected signatures of a chaotic temperature dependence. 

This paper is organized as follows. In section^, we mo- 
tivate and explain this study; our Random-Energy Random- 
Entropy Model is described and solved in section [| In 
section ^, we determine how the equilibrium state changes 
with temperature. In section [s], we generalize this model 
to the droplet/scaling picture. In section [| we discuss the 
predictions coming from another simple spin glass system 



2 



F. Krzakala, O.C. Martin: Chaotic temperature dependence in a model of spin glasses 



based on the naive mean field equations. We summarize 
and conclude in section |t]. 



2 Temperature chaos as level crossings 

Our framework is motivated by the concept of "valleys" 
in glassy systems. Indeed, within the mean field picture of 
disordered systems in their low temperature frozen phase, 
ergodicity is broken and configuration space is broken into 
many components not related by symmetry. These com- 
ponents are often referred to as valleys as in an energy 
landscape picture. Loosely, one considers each valley to be 
associated with one thermodynamic "state" , though such 
an association has ambiguities, especially in finite volume. 
This picture of valleys is not restricted to a mean field 
framework: even in the droplet |Q and scaling [jl3| pic- 
tures, the energy landscape is very rugged and there exist 
multiple valleys corresponding to possibly (meta-stable) 
states that do not contribute to equilibrium properties in 
the thermodynamic limit. We will thus assume that we 
can talk of states (equilibrium or meta-stable) and then 
ask within such a picture what can be said about temper- 
ature chaos. 



Fig. 1. Temperature dependence of free-energies of the lowest 
states, (a): there are no entropic fluctuations, as in the spher- 
ical p-spin model; (b) possible level crossing behavior in more 
realistic models. 



have zero mutual overlap. When following analytically the 
free-energy of these states as a function of temperature, 
one finds that they keep the same order: there are no level 
crossings. 

Although such a property may be special to the spher- 
ical p-spin model, there is also the possibility that it is 
more general, extending to all mean field models or even 
to finite dimensional models; this is supported by the lack 
of any numerical evidence in favor of chaos in either the 
SK model or the Edwards- Anderson model. 



2.1 Level crossings 

In the simple case of the ferromagnetic Ising model be- 
low its critical temperature T c , there are two valleys Vi 
and Vi with degenerate free-energies and magnetizations 
±rn(T). Consider the mean spin-spin overlap q > between 
these valleys: it is q 1 ' 2 = —m 2 (T). The associated (equi- 
librium) states have a smooth evolution with tempera- 
ture, merging when T approaches T c from below. It is also 
possible to consider the overlaps between an equilibrium 
state at T and one at T + ST. Such an overlap is equal to 
±m(T)m(T + ST) , which also varies smoothly with T and 
ST. On the contrary, to have temperature chaos, we want 
such overlaps to be very sensitive to ST, and generically 
one expects the equilibrium states at different tempera- 
tures to be "totally different", meaning their overlap is 
zero (in the absence of an external field) . 

Does one have temperature chaos in disordered sys- 
tems where the abundance of (meta-stable) states plays 
a fundamental role? The answer is yes for spin glasses on 
Migdal-Kadanoff hierarchical lattices flj|,[l5| or for other 
disordered systems like the Directed Polymer in a Random 
Medium in 1+1 dimension []u||r7|] . But in other systems 
one might instead expect that a state dominating the par- 
tition function at T will also dominate it at T + ST. Such 
a scenario is represented pictorially on the left of figure ^ 
where for each state we plot its free-energy as a function 
of temperature. We can think of these curves as a fam- 
ily of levels, and in this case there are no level crossings. 
Interestingly, this is what occurs in the the infinite range 
spherical p-spin model [ fl8| . There one can define states 
in the large volume limit, for instance by the Thouless- 
Anderson-Palmer (TAP) |[9| equations. These different 
states evolve smoothly with temperature, and furthermore 



2.2 The Bray and Moore argument 

To understand why many researchers expect a chaotic 
temperature dependence in finite dimensional spin glasses, 
it is of some use to go over the argument due to Bray and 
Moore (BM) j|, reformulated for our particular purposes. 
For specificity, we work with the 3-dimensional EA model 
with no magnetic field whose Hamiltonian is 

H = - 2J JijSiSj (1) 

<ij> 

The Si are Ising spins on an L x L x L lattice and the 
sum is over nearest neighbor sites with periodic bound- 
ary conditions; the Jij are independent Gaussian random 
variables. We start from two "states" at low temperature, 
differing by a very large droplet and work in the frame- 
work of the droplet /scaling theory of spin glasses ]l3|,[20|]. 
The two states have frcc-cncrgics that differ by AF(T) — 
AE — T AS ~ T£ e , where £ is the characteristic size of 
the flipped droplet when going from state 1 to state 2 and 
T is the temperature dependent stiffness coefficient. Now 
change the temperature; AF(T+ST) « AE-(T+6T)AS, 
so that AF(T + ST) « Ti e - ST AS. 

Following Bray and Moore, it seems inevitable that the 
entropy difference is associated with the droplet 's surface 
so that AS has a random sign and a typical magnitude 
proportional to £ ds / 2 where d s is the fractal dimension of 
the droplet's surface. Then if d s /2 > 8, which follows from 
the inequality 9 < (d — l)/2, the difference in free-energies 
AF(T + ST) cx T£ + ST£ d ^ 2 can change sign between T 
and T + ST. The conclusion is then that the equilibrium 
state(s) between T and T + ST should change on a length 
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scale £ scaling as £(ST) oc ST 1 ^ t ^ while their typical 
overlap qr,T+ST should go to zero as 



Qt,t+st 



<l/2 



(2) 



where L is the total lattice size. In the droplet /scaling 
picture, 8 ss 0.2, while in the mean field picture 9 — 0; in 
either case, the argument strongly suggests that the EA 
model has temperature chaos. At the root of the argument 
are large cancellations between E and S in the equilibrium 
value of F; these cancellations make the equilibrium state 
very sensitive to changes in T. Note that numerical ev- 
idence for such cancellations has been found in the 
3 — d EA model. Finally, as mentioned previously, chaos 
does occur on the Migdal-Kadanoff lattice; 

To us, the Bray and Moore argument is very convinc- 
ing if one goes from T — to T = e for any e > 0, sug- 
gesting that chaos is unavoidable at T = 0. However, the 
argument has potential pitfalls when considering chaos at 
T > 0. First, when going from T to T + ST, both AE 
and AS change, a fact which is neglected in the argu- 
ment. Second, one cannot treat a single scale £ alone; it 
is possible that the droplets on the scale smaller than £ 
will see their free-energy change sign, and thus the "evo- 
lution" of Vi(T) to T + 5T will differ from Vi(T) by many 
droplet excitations. This evolution may be enough to keep 
the free-energy of Vi(T + ST) below that of V 2 (T + ST). 
In this sense, the chaos at scale £(ST) might be quenched 
by the way the equilibrium state adapts to changes in T. 
And since Vi(T) and Vi(T + ST) differ only by droplet 
excitations, their overlap will not be close to zero. Unfor- 
tunately we cannot take into account these effects in a 
realistic way; nevertheless, we want to use the fact that 
the state dominating the partition function at one tem- 
perature is quite special, being the result of an extreme 
statistic where the free-energy is optimized for that tem- 
perature. We now proceed to see how we can model this. 



3 The Random-Energy Random-Entropy 
Model 

In the Bray and Moore argument, the important feature is 
that there are very large entropy fluctuations from state to 
state. We shall now define a glass model that incorporates 
such entropy fluctuations. Since our model is close to Der- 
rida's Random Energy Model |^2| (REM) , we have named 
it the Random- energy Random- entropy model (Re-Rem). 
In the spirit of the REM, we start by taking 2 N "valleys" , 
each with an energy taken from a Gaussian distribution, 
but we shall also assign a random entropy to each valley. 



3.1 Definition of the Re-Rem 

The Re-Rem model is defined as follows. We have 2 N 
states; the free-energy of state i is given by 



Fi — Ei — TSi 



(3) 



where Ei and Si are random independent variables, taken 
from a Gaussian distribution of mean and standard de- 
viation a E = N°- 5 /V2 for the energy and a s = N a /V2 
for the entropy, a being an exponent that will play an 
essential role later. More explicitly, these probability dis- 
tributions are 



Pe{E 1 ) = 



exp 



(-#) 



Ps(Si) 



exp 



( X) 



VN 2a TT 



(4) 



Note that for the Ei, the probability distribution is the 
same as in the REM. A simple way to visualize this model 
is to think to 2 N lines in a temperature-free-energy plot 
as in figure |l| where the intercept on the y axis is random 
as well as the slope of each line. 

Being the sum of independent Gaussian random vari- 
ables, the free-energy has a Gaussian distribution. Thus F 
has zero mean and variance of 1 = <te 2 + as 2 - Explicitly, 
the variance of the free-energies satisfies 



2er F 2 



N + T 2 N 



2,i 



(5) 



If a — 1/2, the two terms contribute the same magni- 
tude at large N, while if a < 1/2, the variance is domi- 
nated by <te- The natural scaling then seems to be a = 1/2 
in which case the ground state energies and entropies are 
both extensive. In a T-F plot, we may expect these states 
to have crossings, like in the right part of figure |]. 

3.2 The equivalent REM 

Because of the Gaussian distribution of free-energies at 
each temperature T, the model is equivalent to a REM 
whose energy variance is given by equation |^. This allows 
one to solve the thermodynamics of the model. The den- 
sity of levels with free energy F is 



Pf(F) = 2 



N ' 



(6) 



As expected, there is a critical dependence on F for large 
N. If \F\ < Fq — CTi?V2A^ln2, there is an exponentially 
large density of states and an extensive configurational 
entropy 



S C (F) = N\n2- 



2a F 2 



(7) 



whereas if |F| > F , there are no levels at all in the 
thermodynamic limit and thus Sc(F) = 0. Using the re- 
lation between configurational entropy and temperature 
T -1 = dSc/dF, the critical temperature is 



T r = 



Up 



V2N In 2 

and thus T c satisfies the self-consistent equation 



y/N + T C 2 N 2 ° 
2ViVln2 




41n2 



(8) 



(9) 
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This lead us to distinguish three cases. First, if a < 0.5, 

then T c = 1/ (2^/la2j, as in the original REM. There is 

a low temperature phase in which the partition function 
is dominated by a finite number of states (those with the 
lowest free-energies); the entropy density in this phase is 
zero. There is also a high temperature phase for which the 
number of states that contribute to the partition function 
is exponentially large in N. Second, for a = 0.5, one finds 
that T c = ,, , 1 „ ; w 0.75. The physics is the same as be- 

V41n2— 1 1 ■' 

fore, but the critical temperature is shifted. In both cases, 
we have a glassy model with a one-step replica symmetry 
breaking transition [^2||2^]. Third, if a > 0.5 one finds 
T c = +oo; there is no phase transition, and the systems 
remains in the "low temperature phase" at all tempera- 
tures. This can be considered to be unphysical because a 
microscopic Hamiltonian such as Eq [j] will always have a 
high temperature disordered phase. If we go back to the 
Bray and Moore argument, in which entropy fluctuations 
grows as l da / 2 , we see that we should identify the Re-Rem 
parameter a with d s /(2d). Happily if one follows the pre- 
scription a = d s /(2d), there is no unphysical behavoir 
because d s < d, so a < 0.5. 

Note that when a < 0.5, the lowest free-energies scale 
linearly with N and the gap between the lowest free- 
energy state and the first excitation is O(l), as in the 
REM. Furthermore if a < 0.5, the lowest free-energy den- 
sity does not change with T, whereas if a = 0.5, it grows 
as yl + T 2 . Finally in the case a > 0.5, the free-energy 
grows as ]\[ a + a - 5 and thus is not extensive. 



3.3 Energy and entropy 

The lowest free-energy in the Re-Rem at each temperature 
is easily found, and from it we can determine the typical 
energy and entropy at each temperature. The lowest free- 
energy at each temperature is given by p2fl 



F a w -y/2Nlog2(T f = — iV^/log 2(1 + T 2 N 2a ~ 1 ) (10) 

The entropy in the low T phase is obtained from the 
derivative of this free-energy with respect to temperature: 



So 



V\og2 TN 2a 
Vl + T 2 N 2a - x 



(11) 



which confirms that the entropy is extensive only for a 
0.5. To determine the energy we use, 



Eq — F + TSq 



(12) 



VI + T 2 N 2a ~ 1 
The energy is thus extensive if a < 0.5. 

4 Level crossings in the Re-Rem 

Given our model with many states and a spin glass phase, 
we focus on the problem of level crossings. We have 2 N 



levels and randomness in entropy and energy. We want 
to follow the lowest free-energy state and see if this state 
changes with temperature, and how often it does if so. 
First, we will provide some analytical constraints, deriving 
a scaling law governing the number of crossings in a ST 
interval; then we will determine the corresponding scaling 
function by a numerical computation. 



4.1 Level crossings via thermodynamics 

At T = 0, the lowest level is the one with the minimum en- 
ergy, entropy plays no role. If one follows the line starting 
from this minimum energy in temperature, we can expect 
some other levels, with larger energies but also more neg- 
ative entropies, to cross this line. This is what happened 
in the Bray and Moore analysis, showing that zero tem- 
perature chaos is inevitable. But what happens at T > 0? 
Does the lowest free-energy state change at each temper- 
ature? What is the density of crossings? The problem of 
knowing which level is lowest at each temperature and 
of computing crossing statistics seems rather non-trivial; 
however thanks to the mapping to the REM, it is possible 
to compute the temperature dependence of the best level, 
and to deduce from that many properties of the crossings. 

Let us begin with some remarks. First, the role of the 
exponent a is just to rescale the temperature definition 
by a factor N a as is evident from equation 3. Thus if 
one solves the problem for a = 0, changing to an a ^ 
just divides each crossing temperature by N a . As a conse- 
quence, the total number of crossings does not depend on 
a. Second, this total number of crossings (the number of 
times the lowest free-energy level changes its slope) is at 
most of O(N). This can be shown using the following ar- 
gument. Take all 2 N states and consider the lowest one £i 
at T = 0. The entropy is random and uncorrelated with 
the energy, so if you take the next state £2 in energy, you 
will have a good chance (50%) that it will have a more 
negative entropy and thus will cross L\. Suppose this is 
the case. A further state Li with a still greater energy will 
cross £2 only if its slope is more negative than that of £2 ■ 
The entropy of a successful candidate Li must be the low- 
est of i entropies taken at random. Since by hypothesis Li 
has the lowest entropy of 2 entropies taken at random, we 
have to let i go up to 4 to have a 50% chance of finding a 
state that will cross £2- Extending this argument, to get 
the k th crossing one will need to examine 0(2 k ) successive 
levels. Furthermore, it is not difficult to show that when 
going from £ 2 fc to £2^+1 the energy increases by O(l). 

Of course, these O(N) crossings are not the end of the 
story because some of them may be irrelevant. Take for 
example the right part of figure [fl where the third energy 
state crosses the second one, but where the fourth one 
crosses the second one before the third does. From this 
simple example we see that not all crossings count; we can 
only say that the number of levels that become at some 
point the state of lowest free-energy is at most O(N), and 
that the energy at each level crossing increases by at least 



0(1). 
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Fig. 2. The lowest states in the Re-Rem remain inside a 
"tube". Left: a > 0.5, the dependence is linear in T; right: 
a < 0.5 the tube is curved. 



In the large N limit, a scaling behavior sets in. But be- 
fore being quatitative, let us see what qualitative prperties 
imply level crossings. The starting point is the fact that, 
at fixed T, the Re-Rem reduces to a REM. The lowest 
free-energy at each temperature scales as in equation [To| 
and its sample to sample fluctuations are of 0(1). Thus 
the lowest state resides in a "tube" (see figure 2) in the 
(T,F) diagram. Clearly, if the tube is not linear in tem- 
perature, there must be some level crossings. We will now 
see how this imposes strong constraints on the crossing 
statistics. 

First, consider the case where a > 0.5. At large N, we 

get 



F oc 



_ jya+0.5 rp 



(13) 



The linear temperature dependence suggests that the tube 
has one, or just of few states passing through it for all T. 
Moreover the slope of this tube is of O(N a+0 - 5 ) which is 
the scale for the lowest entropy value obtained by taking 
2 N values in a Gaussian of variance N 2a . So if at T = 0, 
the best state is the one with lowest E, at any strictly 
positive temperature the best state is the one with the 
best slope. This means that all level crossings take place 
at zero temperature; there is zero temperature chaos but 
no chaos at T > 0! 

When a < 0.5, we have 



Fn « -N^ 



(14) 



Now, the dependence in T is non-linear; there must be 
some crossings at T > because the lowest energy state 
changes its slope in temperature and in N. This suggest 
that the number of crossings in a ST interval diverges with 
N for every temperature. 

Finally, if a = 0, we obtain 

F fa -iV\/ha2 - Vhr2T 2 (15) 

so the dependence on T is N- independent. Since the en- 
ergy increase of successive crossings is at least of 0(1), 
it is not possible to pass an infinite number of levels in 
the tube. Thus there should only be a finite number of 
crossings in each ST interval. 

Now we also know that changing a is the same as re- 
scaling the temperature; this gives a strong constraint on 
the number of crossings in a given interval. Let's assume 



that the scaling of the mean number of crossings in a tem- 
perature interval ST is of the form 



J\f N (T, T + 5T) = N l{a) 5Tg 



Then the invariance under the transformation ax 
and T -> TN ai ~ a2 gives 



j(ai) - 7(0:2) 

k(Q!i) — Av(«2) 



ax — a 2 
a 2 — a\ 



It follows that 



7(a) = a + 0x 
n(a) = —a + (3% 



(16) 



a 2 



(17) 



(18) 



Furthermore, we must respect the constraints derived pre- 
viously. First, assuming g{x) decays quickly (not as a 
power law) at large x, the disappearance of T > cross- 
ings when a > 0.5 gives 02 = 0.5. Second, since a = 
gives crossing density at large N, we must have fix = 0. 
The scaling form for the mean number of crossings in a 
temperature interval 5T is then 



Af N {T,T + ST) = N a 5Tg 



T 



NO-5-c 



(19) 



From this formula, we deduce that the total number of 
crossings is in fact 0(y/~N), much less than the bound 
O(N). Also if g(0) is finite, the case a = leads to a 
mean temperature of the k th crossing that is proportional 
to k. 



4.2 Numerical investigation 

It is of some use to illustrate and check these results by 
a numerical procedure but we also want to compute the 
function g(x). A simple way to do this is to generate 2 N 
random energies, 2 N random entropies, and to check for 
crossings, but such a procedure require a computation 
time that is exponential in N. Instead we used a method 
which allows us to have an algorithm linear in N, and 
thus to simulate up to 2 800 levels. With no loss of gener- 
ality we can focus on the case a = 0; we have simulated 
10000 instances for N = 100, 200, 300, 400, 600 and 800. 

We explain our algorithm in the appendix for read- 
ers interested in algorithmic issues. Basically, our method 
constructs in a time proportional to N the series of cross- 
ing points for a random realization of the Re-Rem. Then 
we measure the statistics of these crossings by averaging 
over many instances, repeating this for different values of 
N. The scaling function g{x) is then obtained by using 
equation [ll]; the results are shown in figure ||. The data 
collapse is excellent, showing that the T j yN scaling works 
very well. We do see some deviations at small N in the 
tail; these correspond to corrections to scaling, and are 
small (invisible) beyond N = 500. 

From this numerical computation of g(x), we can ex- 
tract the number of crossings in any temperature interval 
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Fig. 3. Scaling function g(x) in the Re-Rem. The inset shows 
the raw data for the mean number of crossings versus temper- 
ature for a — 0.5. 



for any a and N. Consider first the case a = 0.5 which is 
the appropriate value in the mean field picture. One has 



A^= (T, T + ST) = VNSTg(T) 



(20) 



Focusing on the interval T c /3 to 2T c /3, we find about 
6 -^/TV/1000 crossings, that is 6 crossings for N = 1000. 

One can also consider other values for the exponent a. 
In the framework of the EA model in d = 3, the fractal 
dimension of the surface of system-size domains has been 
measured [QH), leading to a = d s /2d w 0.46. For this 
value of the exponent the mean number of crossings in the 
T c /3 to 2T c /3 interval is now reduced to about 3 when 
N = 1000. 

The main conclusion of this section is that the number 
of crossings in a temperature interval grows as a power of 
N if < a < 0.5 but with a small prefactor. Thus, if we 
simply map our Re-Rem results to estimate the strength 
of chaos that should arise in Monte Carlo simulations, 
we find that there will be very little sign of chaos until 
N is at least a few hundred. However, the mapping from 
the Re-Rem most certainly over-estimates significantly the 
number of crossings in microscopic models (cf. the next 
section and the final discussion). 



5 Droplet extension of the Re-Rem 

As given so far, our Re-Rem has excitation energies of 
0(1). However, it is possible to modify the model in order 
to mimic the droplet/scaling |ll],|2(J picture of spin glasses. 
There low-lying excitation energies are 0(N e / d ), that is 
0(L e ) where 9 is the stiffness exponent which is at the 
heart of the droplet and scaling pictures of spin glasses. 



configurations. Thus we take a number of states that is 
much smaller that 2 N ; as a consequence, the variance of 
their energy will no longer be O(N). Our method is thus to 
rescale the energies by a factor N e ^ 2d/S> while the number 

of states is reduced from 2 N to 2 N (with j3 < 1). This 
gives 



2a E 2 = N 1+! z 



S c (E)=N' 3 ln2- 



E 2 



2<7 E Z 



(21) 



(22) 



Then T c = a E j^2N^ In 2 and E = -a E V2N? In 2. We 
want an extensive ground state energy at T = 0; thus 



En 



-VNP \n2VN 1+e / d 



(23) 



imposes (3 = 1 — §. It follows that the critical tempera- 
ture is now given by T c = N e / d /2y/\n2, and so T c > 
only if > 0. Note that for 9 > 0, T c — > oo as in the 
Directed Polymer in a Random Medium (DPRM) in 1 + 1 
dimension, in contrast to what happens in realistic spin 
glasses. Actually, this S-REM is very close to the DPRM. 
Consider for instance the exponents giving (i) the finite 
size corrections to the ground state energy, (ii) the sam- 
ple to sample fluctuations and (iii) the energy of the first 
excited state; they are the same (i.e., 9/d) in the two mod- 
els, whereas two distinct exponents arise in the droplet 
model j26|. Nevertheless, this S-REM provides an inter- 
esting picture of the low temperature phase that is quite 
compatible with the scaling/droplet picture there, even if 
it misses T c < oo. 



5.2 Re-Rem revisited 

We can perform the analogous scaling extension for the 
Re-Rem and obtain a Scaling-Re-Rem where low-lying ex- 
citation energies scale as N 9 / d . All the computations are 
straight-forward. One finds that the critical temperature 
of the S-Re-Rem is given by 



^/jsji+e/d + T2 N 2a N 9/d Vl + T 2 N 2a - e / d - 1 



2^/N 1 - <) / d \n2 



(24) 



and that F oc —Ny/l + T 2 N 2a - 1 ~ e / d . Now letting a' = 
a — 9 1 (2d), we have 



N e / d Vl + T 2 N 2a '- 



(25) 



5.1 REM revisited: The Scaling-REM 

To construct the Scaling-REM (S-REM), we note that it 
is not appropriate to identify single spin configurations 
with states, a valley containing a large number of similar 



F oc -Ny/l + T 2 N 2a '- 1 



(26) 



From this, we see that the thermodynamics is the same 
as that of the Re-Rem but where a has been replaced by 
a — 9 1 (2d). We can also apply this correspondence to the 
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formulae for the level crossing properties again again 
that there is temperature chaos. More precisely, if we start 
from the scaling form [l6] then using a! instead of a, equa- 
tion ^ implies as before that (3 2 — 0.5, but f3% ^ since 
the width of the tube grows as N°/ d . However, the num- 
ber of levels being 2^ , the total number of crossing goes 
as V Nf 3 and the general scaling for 9 ^ is 



N N (T, T + 5T)= N a - e / d 5Tg 



T 



jy0.5-a+e/2d 



(27) 



5.3 Criterion for temperature chaos 

Even if the S-Re-Rem model is a bit ad-hoc, it may cor- 
rectly describe some glassy systems at low temperature. 
The general criterion for temperature chaos depends only 
on the value of parameters a and 9/d. If a — 9/d < 0, there 
is no chaos at all, all crossings being pushed to T — oo. 
If a — 0/d = 0, the density of crossings is finite; this is 
what we call weak temperature chaos. IfO < a~ 9/d while 
a — 9/ (2d) < 0.5, there are an infinite number of crossings 
in each temperature interval, that is strong temperature 
chaos. Finally, if a — 9/(2d) > 0.5, there is no finite tem- 
perature chaos because all crossings take place at T = 0. 
Note that if one accepts a — d s /2d, we recover the original 
Bray and Moore argument. However, within our model we 
can estimate the number of crossings, and comparing to 
the discussion of section [|, the case 9 > leads to even 
fewer crossings than when 9 = 0. 



6 Temperature chaos in the naVve mean field 
framework 

Ideally, one would like to confront the Re-Rem predic- 
tions with the chaos properties arising in a realistic model. 
This requires defining finite volume states or valleys and 
following them as a function of temperature. A natural 
approach is to use the TAP equations where each 
state is parametrized by its magnetizations m t = (Si). 
However, these equations have 2 serious drawbacks: (i) 
on the 3-dimensional lattice, the TAP equations fail to 
give non-paramagnetic solutions |27]] ; (ii) even for the SK 
model |2q ] , having N < oo (which is unavoidable numeri- 
cally) leads to unsurmountable convergence problems pjl . 
The source of these difficulties seems to be the strength of 
the retro-action term. An ad-hoc solution consists in ei- 
ther reducing or neglecting this term. Following previous 
authors |^,|3^] , we choose the second option which corre- 
sponds to the naive mean field equations (NMFE) pjj. 



6.1 Naive Mean Field Equations 



The NMFE are 



tanh (P Jijirij) 

3 



(28) 




Fig. 4. P(5F) at different temperatures for < \q\ < 0.05 
excitations in L=12 EA model. Inset: the variance of SF for 
L = 8, 10, 11 and 12 (from bottom to top). 



The solutions to these equations define the naive mean 
field (NFM) states; they correspond to the stationary points 
of the NMF free-energy functional 

F({ mi }) = £({m,}) - TS({mi}) (29) 

where the energy and the entropy functionals are given by 

E = — JijlTliTTlj (30) 

<ij> 



S 



E 



(1 + nii) ln(l + mi) + (1 — mi) ln(l — m,) 



(31) 



1 Note that the width of the tube now grows as jV S//d . 



F has both local minima and saddles; we follow the TAP 
approach and construct in our numerics only local minima. 

Our procedure is as follows. For each disorder instance, 
we will produce two NMF states C\ and Ci and will work 
out their dependence on temperature. We start by build- 
ing their T = intercept: C\ is in fact the ground state, 
while £ 2 is a large-scale low-energy excitation. (To ob- 
tain the ground state, we use a state-of-art algorithm [32|. 
In virtue of the method used to generate excitations [ 24 \ , 
the excitation energies are 0(1) and do not grow when 
increasing N.) We then numerically determine how the 
Ci evolve as we increase the temperature from to T c . 
In practice, we increase the temperature T in sufficiently 
small steps (AT = 0.05) and perform steepest descent on 
the free-energy functional, using as starting point Ci at 
the previous temperature. With this procedure, referred 
to as heating of the ground state in p0| , we consider two 
NMF states that evolve with increasing temperature. 



6.2 Numerical results for the 3 — d EA model 

First, let us give some qualitative properties of this NMF 
system. We find a non-zero Edwards- Anderson order pa- 
rameter qEA = mi 2 /N for any instance and any state 
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when T < T c (T c «5 in the 3 - d NMF EA model, and 
T c «2 in the NMF SK model), and qEA is self-averaging. 
If we follow a given state, the two temperature spin over- 
lap q T = J2i m^ T=0 ^mi^ /N decreases and reaches zero 
when T = T c . Furthermore, the equal temperature mutual 
overlap between C\ and £2, Qtt = 

also decreases smoothly when going from T = to T c , sug- 
gesting that evolving those states according to the NMF 
equations mainly just reduces the size of the m, =< Si >. 
Do we get level crossings as would be expected if there 
was temperature chaos? We are particularly interested in 
the behavior of state to state free-energy fluctuations at 
very low temperatures as this quantity is at the heart of 
our Re-Rem picture. 

For this model, we have used on average 1000 ground 
states and excitations for sizes L = 8, 10, 11 and 12. We 
focus in all that follows on the instances where our exci- 
tations have an overlap with the ground state satisfying 
< \q\ < 0.05. However the data for other overlap inter- 
vals have a qualitatively similar behavior. 

According to zero-temperature chaos, when going from 
T = to T > 0, the valley L\ should be above the valley 
£2 with a strictly postive probability when TV grows. We 
find this to be the case, and have found that the prob- 
ability to find such an inversion increases with tempera- 
ture. To understand this more quantitatively, define SF 
to be the free-energy difference ^(£2) — Figure || 

shows the distribution P(SF) at different temperatures 
for L — 12. The variance of this distribution increases 
with temperature. The curves for different L and differ- 
ent window overlaps look similar, so once T > 0, there 
is a positive probability that SF < 0, corresponding to a 
level crossing. Since there are an infinite number of such 
excitations in the thermodynamics limit, and since they 
plausibly all behave the same way, these crossings suggest 
that there is indeed temperature chaos. 

To make contact with our Re-Rem picture, one should 
extract a from the L dependence of the variance of SF. If 
a > 0, the variance of SF will diverge at large L as L 2ad . In 
the inset of figure ^ we plot the variance of SF at different 
temperatures. At any given temperature, we can fit its L 
dependence; not surprizingly, just as was previously found 
at T = ||, H), the data can be fit with a = 0.5 (the 
mean field case d — d s ) while the best fit gives a w 0.46 
(d s w 2.76). What is important is that a is certainly not 
0. This suggests that this system have strong chaos. 



6.3 Numerical results for the SK model 

Recently, the naive mean field framework was used by 
Mulet et al. 1 30 1 to study temperature chaos in the SK 
model. Although they concluded that there is no chaos, 
we feel it is necessary to reconsider that conclusion in view 
of the Re-Rem analysis. 

Compared to the EA model, the SK is numerically dif- 
ficult because of its long range interactions. More numer- 
ical effort is needed to find ground states or excitations 
and also for doing the steepest descent on the NMF free- 
energy functional. We were thus limited to "small" values 



0. 4 



0.2 



T=0 
T=0.5 
7=0.75 
L=1.0 





■0.2 



0.2 



0.4 



Fig. 5. P[5F) at different temperatures for < \q\ < 0.05 
excitations in the SK model (TV = 192). Inset: the variance of 
SF for L = 64, 128 and 192 as a function of T (bottom to top). 



of TV: we used TV = 64, 128 and 192, performing statistics 
using 50 ground states and excitations for each value of 
TV. 

Our conclusions are qualitatively similar to those we 
obtained for the 3 — d EA model (see figure . We find 
that there is a strictly positive probability that C\ and £2 
cross; fits of the free-energy fluctuations give a value for a 
fully compatible with 0.5. These results are those that are 
expected in the Re-Rem picture; it is thus appropriate to 
conclude that there should be temperature chaos. However, 
note that the variance of SF grows only slowly with TV so 
very few crossings are expected until much larger values 
of TV are reached. 



7 Discussion 

This work has two main points. First, we have developped 
a solvable model of spin glasses inspired from the REM 
that allows a precise study of temperature chaos. This 
model has strong temperature chaos unless its exponent 
a is artificially set to zero. However; this chaos is absent 
until the system size TV reaches about several hundred. 
Second, we have argued that this Re-Rem model should be 
relevant for real spin glasses by considering the particular 
case of naive mean field equations. The numerical study 
of that system turned out to be in complete agreement 
with the expectations of the Re-Rem picture, giving strong 
evidence that temperature chaos does occur in spin glass 
models such as the 3 — d Edwards- Anderson and the SK 
models. 

The question of temperature chaos has lingered on for 
many years because of lack of numerical evidence. One 
may considering the absence of chaos signals in past and 
present MC simulations by the fact that in the Re-Rem, TV 
must be quite large for chaos to appear. Indeed, we saw 
that in the Re-Rem chaos becomes clear for TV ~ 1000 
but it is clearly inappropriate to consider that one has 2 N 
states in a physical system of TV spins. Having far fewer 
states will lead to fewer crossings, and so it may be that 
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no chaos will be seen from MC simulation in the near fu- 
ture. However, we believe that our naive mean field study 
suggests on the contrary that chaos in realistic systems is 
within grasp of current Monte Carlo techniques. One way 
to do this is to consider for instance the 3 — d EA model 
and to extract at each temperature the states or "valleys" 
by clustering the equilibrium configurations. Then follow- 
ing these states in temperature will allow one to compare 
directly with the Re-Rem picture. 

In the future, it would be interesting to extend the 
Re-Rem to more complex systems such as the Generalized 
Random-Energy-Model. In such a G-Re-Rem model, one 
would be able to understand the effects of 2-step, 3-step 
and continuous RSB. Another interesting possibility is to 
study the Re-Rem with more realistic behavior than a 
pure linear dependence in T, the free-energies of levels 
being a more complicated function of T. 
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Appendix 



Here we explain in detail our algorithm. We want to 
construct efficiently the series of crossing points for a re- 
alization of the Re-Rem having 2 N states or levels, level 
i having energy Ei and entropy Si. It is convenient to 
assume that these states have been ordered in increas- 
ing value of their energies. We will start with the ground 
state: this level 1(0) = has energy-entropy (^(o), 5/(o))- 
Then we will continue by finding the next level 1(1) that 
has a lower entropy than that of ^(0), i.e., S/m < S^ Q y 
This construction is extended by recurrence: given the 
level number l(n) and values (£/(„) , S/(„)), we will gen- 
erate with the correct probability distribution of the next 
level l(n + 1) that has S^ n+ i) < S/( n ) as well as the quan- 
tities (£/(„ + i), Si( n+1 )). The key point is that all these 
quantities can be generated recursively without "looking" 
at all 2 N states. Once this list of (l(n), E^ n y S^ n y) values 
is constructed (this takes 0(N) operations), we can then 
reconstruct the lowest state at any temperature, and thus 
the statistics of the crossings. 

To begin our algorithm, we set n = 0, 1(0) = and 
choose <S/(o) according to its Gaussian distribution. Then 
we must generate -E;(o) (note that -E;(o) and S^q) are un ~ 
correlated random variables). The probability density of 
E m is 

P(E m ) = 2 N p N (E m )[l - Q(E m f N - l) (32) 



where pn(E) is the probability density of energies in the 
Re-Rem model (a Gaussian) and Q is defined by 

/E 
PN (x)dx (33) 
-oo 

To generate E t ^ with its distribution, we use the ac- 
cept/reject method. Finally we set E D u = £Wo) and S u = 
Si( )- 

Now we enter the recursion. Given the values l(n), E Q u 
and S id, we first determine the next level l(n + 1) that 
has an entropy below S Q id- Let Al be defined by l(n + 
1) = l(n) + Al; Al is exactly the number of times one has 
to generate a random entropy S new according to the Re- 
Rem entropy distribution until we find a value satisfying 
Snew < Sold- Define p to be the probability to get a slope 
smaller than S Q id with one trial; then 

P= [ S ° ld PN (S)dS (34) 

J — oo 

Clearly, Al (the number of attempts made before reaching 
an entropy value lower than S id) is a random variable; its 
probability distribution is 

P(Al) = (l-p) Al ~ 1 p (35) 

We thus generate Al according to this distribution. This 
gives us the new level to consider, l(n + 1) = l(n) + Al. 
If l(n + 1) > 2 N , we stop as such a level does not exist. 
Otherwise, l(n + 1) is a legitimate level, and we proceed 
by determining (E^ n+1) , Si( n+1 )). 

The entropy of level l(n + 1) is easy to generate be- 
cause its distribution is that of any Si subject to the con- 
dition that S new < Sold- The only difficulty is for gen- 
erating Ei( n+ iy To do that, start with P(E Q id, E new ) = 
P(Eoid)P(Enew\E id) where the quantities l(n) and l(n + 
1) are considered as known. From this relation, we derive 

P(F ip 1 ,,")— f + 1) — l(n) — 1 \ 
P(E new \^old) - ^ M _ (z(n) + y J 

x Q(E old ,E new ) Al ^ 1 (M - l(n+l))p N (E l[n+1) ) 

(l-Q(g K „ +1) )) M -('("+D+i) 
(l-Q(E l(n) )r-H(n) + i) W 

when M = 2 N and Q(a, b) = Q(b) - Q(a). We sample this 
distribution by the accept-reject method again. 

Equation 35 has a simple interpretation; we have built 
a set of "volumes" in the space of EiS. The relative space 
we can explore now has weight [1 — Q(Eir n \)] M ~^ n>>+1 \ 
In this space, we first generate Al — 1 energies in the in- 
terval [-E;(„), -E/(„_i)] and this gives the first part. Then 
we assign the level whose energy is Ei( n+1 y There are 
M — l(n + 1) ways to choose the level among the remain- 
ing ones. Finally, the last (M — (l(n + 1) + 1)) energies 
have to be assigned and then we must normalize the dis- 
tribution. Given this E newi we set -E/(n+i) to E new and 
we are finished. To continue the recursion, we set E u to 
E new , Sold to S new and nton+1. 



10 



F. Krzakala, O.C. Martin: Chaotic temperature dependence in a model of spin glasses 



To summarize the whole procedure, using these prob- 
ability distributions, we can directly create an instance of 
a list of crossings by producing the O(N) £)(„) and «Sj( n ) 
that will cross and then study these crossings (this is a 
simple problem) . From a numerical point of view however, 
these distributions are not so easy to compute, and one 
has to take care of machine rounding errors; that source 
of error limited us to N < 1000. 
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